Subject: Technical Q+A by Ray Bird and Lap Huynh

From: "Marcia Peters (8-444-4380)" <[email-masked]>
Date: Mon, 31 Oct 94 10:22:58 EST

I'm posting Ray and Lap's answers to questions
discussed during the Oct. 13th SIG meeting.

Marcia
---------------------- Attachment -----------------

The following are all the comments received and discussed at the Oct.
AIW.  The resolutions (or non resolutions) are indicated by '*****'.

Ray and Lap

*************************************************************************
*****          Comments from the AIW HPR SIG meeting                *****
*************************************************************************
- Why don't you add a section that states what the APPN dependencies are
  (like segment reassembly support required in ENs).

  ***** Agree - will do.

*************************************************************************
*****   Comments from Data Connections (Mathew Finlayson)           *****
*************************************************************************

Hi there,

We've finally collated all our feedback on the AP draft of the HPR
specification.  Apologies for the delay, but the arrival of the spec
clashed with the late summer holiday season.

Firstly, the specification is vastly improved from the AD level,
particularly the RTP sections, and looks to be a very good basis for
requesting AP status.  Well done to all involved.

Aside from the detailed (and minor) technical and editorial points below,
the only significant issues we see in the specification are:

-   it has little description of how FID5 path control interfaces with
    and/or changes the existing APPN architecture

    ***** This is internal node structure work.  We will include an
          overview picture of the APPN node structure modified to
          contain HPR.

-   it has no network management content (eg. Alert and MIB information).

    ***** This is a separate parallel work effort.

Therefore, we propose that, although these points should be addressed
before CP status, they should not delay the draft achieving AP status.

Technical points:
=================

SHORT_REQ timer
---------------

Firstly, a comment on the answers you sent to our last batch of questions.
You wrote:

>*** The SHORT_REQ timer does not have to be accurately initialized
>    since it will be adapted to the path after a few status requests
>    (approximately 6 samples, see awp).   It is initialized to 1 second
>    (i.e.  a relatively large value).   It can be estimated based
>    on the accumulated transmission time but this information does not
>    include propagation delay so it must be accounted for.   The
>    architecture specification chooses the former method to initialize
>    this timer.

Thank you.  We found during our initial live testing that it is necessary
to start the SHORT_REQ timer at a large value.  In fact, to allow for debug
code (tracing etc) on the destination and intermediaries we were using a
figure of around 5 or 6 seconds.

***** This is fine for debug but when the product is shipped by sure it is 1.

Section 7.2.5.1.3.2 (SHORT_REQ timer) ought to be expanded to discuss the
initialisation of the SHORT_REQ timer.

***** Agree - will clarify documentation.

However, I'm not sure about the convergence rate you suggest (and we don't
have the awp you reference).  Using the formula for smoothed round trip
time provided in the spec, we get the following figures for convergence of
the SHORT_REQ timer when initialised to 1 second.

  Actual RTT        Number of iterations to               Smoothed RTT after
                    reach within 100% of actual RTT       6 iterations

  100ms             17                                    > 500 ms
  10ms              35                                    > 450 ms
  1ms               52                                    > 449 ms
  100 microseconds  69                                    > 448 ms

***** Sorry for the confusion.  The convergence rate mentioned is for
      the case when the initial SHORT_REQ timer is actually smaller
      than the actual RTD (Rountrip Delay) causing premature time-outs.
      When this happens, the SHORT_REQ will adapt to the new RTD in
      roughly 6 samples.  If the initial SHORT_REQ timer is longer
      than the actual RTD, there won't be any time-out and SHORT_REQ
      can take longer to converge to the actual RTD.

Status-Request
--------------

On pages 7-18 and D-21 the specification now suggests that the SR flag
should be set whenever the RTP connection's send queue become empty.

Our intial (admittedly brief) analysis suggests that this may cause the
send-rate to oscillate unduely, particularly if the offered load is
slightly below the saturation level.  Surely this is undesirable?

In fact on any connection where the offered load is below the current
send-rate the SR bit will be set on every packet sent, with the
accompanying overhead as discussed on 7-18.  This would certainly be
undesirable on any connection where the charging is performed on a
per-packet basis (for example X.25).

***** We can't think of a better solution - any suggestions are welcome.
      We must guard against the case where the message that is sent is
      the last (e.g. has CD).  If status is not requested on this message
      and it is lost the session is hung.

ALIVE timer
-----------

The ALIVE timer is the only way a hung connection will be detected in some
scenarios (for example, if the single session on the connection is in
receive state, and the partner node abends) if the option to scan for
topology changes is not implemented.  This suggests the default value of 30
minutes for this timer (page 7-21) is far too long.  We suggest that the
default value be 45 seconds (ie. the same as for limited resource links),
since this will give a worst case detection time similar to that on LANs
today.

***** This is a difficult one.  Mathew pointed out a case where it would be
      very desireable to have a smaller liveness timer value.  A stationary
      node sends a message to a mobile partner but a wired link breaks and the
      message doesn't get to the mobile node.  Since the stationary gets
      no acknowledgement, it goes into a passive path switch state waiting
      for the mobile node to initiate the switch.  However, the mobile node
      doesn't initiate a path switch because it doesn't know anything is wrong.
      The only way the mobile node finds out something is wrong is via the
      liveness message.  Therefore, this is a case where it would be desireable
      to have a liveness timer value (much) less than 30 minutes.  This item
      will continue to be worked on and resolved over e-mail.

Route Setup Backout
-------------------

The mechanisms for route setup backout do not discuss what should happen
during path switch.  Clearly backout is not appropriate while an existing
connection is attempting to path switch.

We can see two possible ways for preventing it

- Intermediate nodes pay attention to the "Route Setup for path switch"
  bit on the Route Setup GDS variable and do not perform backout if it
  is set.

- The origin node checks to see if backout has occurred and if so discards
  the route and looks for another.

We would suggest both of these approaches are used.

***** A destination node must not do a backout on a Route Setup for path
      switch.  However, if it (erroneously) does, the origin node will
      reject as you suggest above.  Intermediate nodes will not make any
      checks.  This will be clarified in documention.

Route-Selection for HPR
-----------------------

Section 7.8 (on page 7-46) and Chapter 14 should probably reference each
other.  Chapter 14 could include a warning about the overuse of the HPR
specific COS's, potentially causing "network collapse".

***** The whole idea of having HPR default COSs was questioned.
      It was suggested by Dave Bryant that we just use the mechanism
      already in HPR for calculating HPR only routes on path switch.
      Each COS would indicated whether HPR was preferred or not and
      the route would be calculated accordingly.  We will look at
      this idea and propose a final solution.

      No matter what the solution, the "network collapse" problem
      will be documented.

The specification should make it clear whether the implementation of the
HPR specific COS's is optional or mandatory.  It should also discuss the
impact on the NOF DEFINE_COS interface - presumably we must allow users to
define COS's which are affected by the HPRness of a link.

***** Support of HPR default COS's is optional.  User defined fields may be used
      for associating HPR properties with a link.  Products already support
      this and nothing beyond this is required.  See above comment on new idea
      for COS.

In section 7.8.1 the specification states that in a network consisting
entirely of HPR nodes the HPR-only tree and the normal session tree will be
the same.  Presumably the spec means identical since there is no algorithm
for TRS to realise that it does not need to generate the two different
trees.  Indeed - the trees may well differ, since they may have been
randomised differently.

***** Agree - will document accordingly.  If all nodes/links are HPR then
      only one tree need be kept.

Use of a single RTP connection
------------------------------

The paragraph at the bottom of page 7-7 reads as follows:

> All traffic from one session flows over a single RTP connection, but many
> sessions may be multiplexed onto one RTP connection.  All sessions between
> two nodes that have the same COS and use the same path are transported
> over a single RTP connection between the HPR nodes containing the session
> endpoints (or BFs).

It would make sense in some circumstances to allow a node to bring up more
than one RTP connection for the same COS and same path.  In the scenario
where only a single path between two nodes exists, but they are bringing up
many sessions (on the same COS), the use of multiple RTP Connections would
allow them to gain more bandwidth for the sessions.  This is important,
given that ARB works to produce fairness among RTP connections, but the
user's perception of fairness is tied to sessions.

***** Agree - will document.  It is not recommended that products use multiple
      RTP connections for the purpose of obtaining more network bandwidth
      in an unfair manner.  This is tricky and will be documented.

Compressed TG descriptors
-------------------------

Is this an APPN architectural change being proposed elsewhere, or just HPR
specific?  (If the former, where is it documented?)

***** It is an architecure proposal in the APPN Reference SIG.

On page 9-17, the specification should make a recommendation for building
RSCVs.  (Should the CV46 contain both the 80 and the 86, or just the 86?)

***** We recommend putting in the 86.  This will be documented.

NCEs
----

The NCE of the LU originating a directory search is now supplied on a
directory search with the FIND (see section 8.3.1 on page 8-3).  What is
the purpose of this?  There seems to be little point in caching it at the
remote end, since it will have to do a directed search (which will obtain
the NCE again on the FOUND).  In fact the information is fairly
meaningless, since the origin LU may not have any sessions active, which
means that the NCE may be changed at any time.

***** The NCE is on the FIND for future use (e.g. SLU initiate).  Will
      be documented.

In the discussion of NCEs the specification makes it clear that it is valid
to use one NCE for all LUs and BFs.  What it does not specify is whether it
is legal to use one NCE for all purposes in a node.  Is it legal to use the
same NCE value for all NCE_LUs, NCE_BFs, the NCE_CP, and the NCE_RS?

***** Yes, it is legal to have one NCE per node (will clarify document).

On the same topic, can you make it clear whether the NCE_CP is equivalent
to the NCE_LU for the CP LU.  Section 7.1.2.5 on page 7-5 hints that this
may not be the case (ie. NCE_LUs can change but the NCE_CP must not).

***** NCE_CP is the same as the NCE_LU for CP LU.  Will clarify that NCE_LU
      for CP LU cannot change over the life of a link.  Once the CP NCE is
      communicated to the partner via XID, it cannot change as long as that
      link remains active.

Link-level error recovery
-------------------------

On page 4-2 the paragraph entitled "Link-level non-ERP support" should be
clarified.  It is permitted to specify that you require HPR ERP in the XIDs
exchanged on a link (and if the other end cannot provide this, then the
link cannot support HPR).  This paragraph seems to suggest the opposite.

> The ability to send packets over a link without using error recovery
> is required support.

***** For X.25 and channel, link-level error recovery (ERP) is required.
      Not doing ERP is not an option.

      For all other link types (Frame Relay, LAN, SDLC) not doing ERP
      is required.  Doing ERP is optional for these links.

      The reason for the above requirements is to guarantee interoperability
      between all HPR products because they can always be configured to
      use the ERP in the required way.

On the same subject, a question about X.25 support.  The specification
(Section 6.2.4, page 6-4) explains that the QLLC option of X.25 is used
when supporting HPR traffic.  What should the ERP settings be for XIDs
exchanged across an X.25 link contain?  (Should they require ERP or require
no-ERP?)

***** X.25 always uses ERP because that's what the X.25 network (and
      access links) will do regardless of what's specified on XID.

In section 6.3 (page 6-5), the specification mentions the settings of the
'"send" timer' and the 'number of retries' on a link.  Since the section is
discussing "Link failure detection when not using error recovery", these
two link parameters are not applicable and reference to them should
probably be removed.

***** The "send" timer and the "number of retries" are required to detect
      link failure (i.e., they are used when sending the "link liveness"
      message).

Editorial Points:
=================

Data-length
-----------

In the formats section of the specification on page A-9, the DATA length
field is described thus

> Data Length Field (DLF): The exact number of bytes carried in the DATA
> field.

whereas on page D-27, in the discussion of the RTP protocol, it is
described as follows:

> User Message Length: Number of user message bytes in the data payload
> including the End-of-Message character (if present)

However, since the End-of-Message character is only a conceptual thing,
these two descriptions seem to contradict.  Which is correct?  We are
assuming that the first is what is intended, but the references should be
clarified and the terminology made consistent.

***** DLF is correct.  Will fix appendix D to match.

RASAP flag
----------

The specification is still confusing in this area.  The Formats section
documents that HPR always sets this flag, but Appendix D contradicts it.
This should be clarified further.  Chapter 7 should probably mention that
HPR makes use of RTP in this way and Appendix D should explain that some
RTP users do not make use of the RASAP mechanism.

***** Agree - appendix D will be updated.

ARB
---

The documentation of ARB on pages 7-32 and 7-33 does not discuss the
feedback type of RESTRAINT.

***** Agree - will fix.

Priority
--------

In section 7.4 (on page 7-46) is might be desireable to add a clarification
explaining that NLPs containing BINDs flow at the same priority as the rest
of the connection.  (We are aware that this is already mentioned
elsewhere.)

***** Agree - will add note to this section.

CRSS
----

We assume the Composite RSCV control vector X'85' (CRSS) referred to on
page 12-1 and the Composite Route Selection Subfield (CRSS) X'4685'
documented on page A-31 are the same thing?  If so, the terminology should
be consistent.

***** Yes, they are the same.  Will clarify documentation.

Segmentation
------------

The specification implies in several places that all "not EOM" packets
should be MLS-size (specifically on page 7-24).  This is not true.
Counter-examples include:

- an empty packet which just transports THDR information

- a retried packet which has to be retransmitted with a larger THDR

- a retried packet which has to be retransmitted after a path-switch

***** Don't agree - but will document anyway.

Minor corrections:
------------------

A few minor or typographical problems we've spotted:

(PEDANTIC MODE ON)

- Annotation 1 on page 9-8 wrongly refers to the old mechanism for
  Long-Lived RTP connections (which were brought up ahead of time at
  link-activation time).

***** Agree - annotation will be fixed.

- Pages 7-32, 7-33 and 7-41 are missing Theta characters in the text.

***** Whoops - will fix.

- Page 9-16 contains the word "receiveg".

***** Will fix.

- The document uses "it's" throughout when it ought to be "its".

***** Will fix.

- Sense code A0010037 refers to the setting of the Window flow control
  bit.  Since this is no-longer documented in the specification, the
  sense code should either refer to the appropriate reserved bit, or
  be removed.

***** Will fix.

- Reference to "customers" has crept back into the new Chapter 14. As
  discussed in earlier emails, this should be corrected.

***** Will fix.

- There are font problems in the headings of Appendix A.

***** Will try to fix (not sure what the problem is).

(PEDANTIC MODE OFF)

*************************************************************************
*****      Comments from Insession (Dave Finnie)                    *****
*************************************************************************

All,

I have not yet completed my review of the HPR spec,
but here is what I have found so far:

(BTW, my PEDANTIC MODE is infuriatingly stuck on the
"ON" position :-))

* p6-3:

  The calculation of the BIND PIU data length is incorrect.
  The assumption is the use of a FID2 (6 byte) TH yielding
  6 + 3 + 512 = 521. Since we are talking about NLPs here,
  FID5 should be used, yielding 12 + 3 + 512 = 527.

  ***** Agree - will fix.

* p7-5:

  (Following on from Matthew's remarks.) The indication
  is that RS and CP NCE values may *never* change. Is this
  in fact never, or just for the life of a link ? If never,
  what is the proposed recovery algorithm if a node
  needs to be cold loaded. If for the life of the link
  (or other criterion) please indicate.

  ***** The RS and CP NCE cannot change during the life of the link.
        Will clarify documentation.

* p7-16:

  In the FSM, the description of state 4 should have
  the word "not" removed. Alternatively, the replace
  word "Partner" with "This".

  ***** Agree - will correct.

* p7-21:

  I agree with Matthew's comments that a 30 min default
  value for the ALIVE timer is too high.

  ***** See discussion on Mathew's question about this.

* 7-24:

  Segmentation and Reassembly. Forgive me if I have
  missed something here, but...How do you envisage
  RTP processing fitting into the node ? Specifically,
  would a new component exist that implements RTP
  functions ? That is my impression. If so, where does
  it fit in relation to PC and how does it interact ?
  Is PC a user of RTP ?

  What I am leading to here is: should PC segmentation
  + RTP segmentation be used together (i.e. RU
  segmentation into FID5 PIUs + FID5 PIUs into NLPs).
  There would seem little point. In addition, this
  could quite often produce poor performance, since
  RTP segments will often have smaller data segments
  than PC's knowledge of the link capabilities
  (since intermediate nodes in the RTP connection
  may have smaller link capabilities). Therefore
  each (large) PIU may be segmented into a reasonably
  large NLP + a small one.

  Could you please include a small paragrap or two
  describing the interaction, and if I am right, an
  indication that PC segmenting should be disabled
  over RTP connections ?

  ***** See section 10.4.4 on page 10-11.  It has exactly what you want.

* p7-32:

  Indicate "GREEN" is used if feeback is "RESTRAINT".

  ***** Agree - will fix.

* p7-34: (near bottom of page)

  Rs = min(0.5 x Rs) is missing the second arg to
  min() function.

  ***** Agree - will fix.

* pA-9:

  Discussion of a DATA field that is N bytes long, starting
  at sequence number x, should include sequence numbers
  x, x+1, ..., x+N-1 (not x+N). Similarly, the subsequent
  data field starts at sequence number x+N (not x+N+1).

  ***** The description for the BSN field is correct.  We will further
        clarify it by explaining that the special EOM character is included
        as part of the sequence number.

* pA-29:

  The description of the length byte indicates that it
  includes the "reserved byte", but this byte appears no
  longer reserved (contains subarea route indicator).
  Also, bit 0 is the subarea route indicator, but so are
  bits 1-7. Is this just a typo and 1-7 supposed to be
  reserved ? If not, bits 1-7 need a different name and
  a description.

  ***** Agree - will fix.

Having picked up on what you will notice are fairly trivial
points, congrats to all who contributed. It is a great
improvement over the original doco.

Dave